Automatic Diagnosis of Student Programs in Programming Learning Environments

This paper describes a method to automate the diagnosis of students' programming errors in programming learning environments. In order to recognize correct students' programs as well as to identify errors in incorrect student programs, programs are represented using an improved de­ pendence graph representation. The student pro­ gram is compared with a specimen program (also called a model program) at the semantic level after both are standardized by program transformations. The method is implemented using Smalltalk in SIPLeS-II, an automatic pro­ gram diagnosis system for Samlltalk program­ ming learning environments. The system has been tested on approximately 330 student pro­ grams for various tasks. Experimental results show that, using the method, semantic errors in a student program can be identified rigorously and safely. Semantics-preserving variations in a stu­ dent program can be eliminated or accommo­ dated. The tests also show that the system can identify a wide range of errors as well as pro­ duce indications of the corrections needed. This method is essential for the development of pro­ gramming learning environments. The tech­ niques of the improved program dependence graph representation, program standardization by transformations, and semantic level program comparison are also useful in other research fields including program understanding and software maintenance.